Various techniques, called a super-resolution processing, for generating a high-resolution image from a plurality of low-resolution images have been developed. Representative super-processing methods which have been proposed include an ML (Maximum Likelihood) method, an MAP (maximum a posteriori) method, and a POCS (Projection Onto Convex Sets) method.
The above-mentioned methods are kinds of super-resolution reconstruction processing. In these kinds of processing, a high-resolution image is successively updated by iterative computation, based on a degradation model for relating a high-resolution image with a low-resolution image, to obtain a high-resolution image which reproduces the original image more faithfully. In the iterative computation, the high-resolution image is updated in such a manner that the differences between a pseudo low-resolution image (degraded image) obtained by degrading a high-resolution image by use of a degradation model, and a plurality of low-resolution images obtained by observation are minimized.
In the above-mentioned iterative computation, a high-resolution image which serves as an initial value (initial high-resolution image) needs to be generated in advance, and input. Depending on the closeness of the initial value to the original image, the number of times the computation needs to be iterated for obtaining a high-resolution image with a sufficient picture quality, or the picture quality that is ultimately obtained varies.
A generally known method for obtaining an initial high-resolution image is to enlarge, by interpolation, an arbitrary frame of a low-resolution image. Another method is to project (perform registration of) pixel values in a plurality of low-resolution images having shifts in the positions at which the pixel values are sampled (having position shifts), onto a corresponding high-resolution space according to the position shifts, so that they are associated with the pixels in the high-resolution image, thereby to obtain an initial high-resolution image with a high picture quality.
When an initial high-resolution image is generated using the above-mentioned method of registration, the high-resolution image may contain pixels (undefined pixels) with which no pixel in the low-resolution images is associated, and of which the pixel value is therefore not defined.
To solve this problem, patent reference 1 proposes to estimate the pixel values of the undefined pixels from the neighboring pixels, or to enlarge an input low-resolution image which serves as a reference and to fill the pixel values of the undefined pixels by referring to corresponding pixels in the enlarged image, or to obtain the pixel values of the undefined pixels by alpha-blending the pixels estimated in the above-mentioned methods.